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(54) A multimedia system with improved data management mechanisms 

(57) A digital multimedia newsroom production sys- 
tem allows users of the system to create, browse and 
catalog multimedia assets. The system includes a mul- 
timedia capture and encoding system that captures 
multimedia data, and substantially simultaneously pro- 
vides a first compressed version of the multimedia data 
having a first resolution, and a second compressed ver- 
sion of the multimedia data having a second resolution 
that is different from the. first resolution; a multimedia 
storage system, coupled to the multimedia capture and 
encoding system, that stores multimedia information 
including the first and second compressed versions of 
the multimedia data; and a video editing and playback 
system coupled to the multimedia storage system. The 
video editing and playback system includes editing cir- 
cuitry that generates a composition that uses a portion 
of the first compressed version, and playback circuitry 
that plays the composition using a portion of the second 
compressed version that corresponds to the portion of 
the first compressed version. The multimedia storage 
system stores multimedia information including the 
compressed version of the multimedia data, and pro- 
vides to a network the first compressed versions of the 
multimedia data substantially simultaneously as the first 
compressed version is stored. 
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Description 




Field of the Invention 

[0001] The present Invention relates generally to a 
multimedia system with improved data management 
mechanisms, and more particularly to a method and 
apparatus for substantially simultaneously encoding 
multiple versions of a multimedia data signal, and pro- 
viding substantially simultaneous access and storage of 
the multiple versions, a correspondence between the 
multiple versions being generated during storage. 

Background of the Invention 

[0002] Over the last few decades, the process for 
producing broadcast news programs has undergone 
several changes. Increased competition brought about 
by the expansion of cable outlets and other news 
sources, and changes in technology, have forced news 
broadcasters to use their resources more effectively. 
[0003] To produce a news program, a typical news 
production organization performs four major operations, 
which are illustrated in FIG. 1. In particular, the opera- 
tions include video production 10, graphics production 
12, text production 14 and on-air operations 16. Unfor- 
tunately, the results of these operations rarely are com- 
bined effectively until the actual broadcast of the news 
program. 

[0004] Video production 1 0 includes generating and 
editing motion video for broadcasting using video infor- 
mation retrieved from a video archive or produced from 
various sources (e.g., cameras, either studio or field 
recorded). Text production 14 includes scripting and 
editing of text gathered from several sources including a 
text archive. Similar to video production 10 and text pro- 
duction 14, graphics production 12 includes generating 
and editing graphics data, such as titling and still 
images gathered from a variety of sources. 
[0005] In order to produce a final news product for 
broadcast, results from video production 10, graphics 
production 12 and text production 14 must be properly 
integrated during the on-air operations 16. Existing 
news broadcast systems are capable of such integra- 
tion. In particular, these systems permit complete man- 
agement of the audio and video elements of the news 
program from acquisition, through editing, distribution 
and on-air play 

[0006] A conventional process for integrating the 
major operations is illustrated in FIG. 2. As shown in 
FIG. 2, a disk-based video production operation 30 is 
integrated with a media production process 32 and on 
air operations 34. The use of disk-based digital 
audio/video storage systems, digital networks, and dig- 
ital non-linear editing systems has allowed for success- 
ful integration of video production, graphics production 
and on-air operations. Several products are available 
from Avid Technology, Inc., Tewksbury, Massachusetts, 
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for providing the wBgration process shown in FIG. 2. 
[0007] The newsroom text production and manage- 
ment system 14 of FIG. 2 is the same text production 
and management system 14 shown in FIG. 1. Although 

5 newsroom computer systems have been in use for sev- 
eral years, these computer systems are predominately 
text based, and have limited integration capabilities with 
tape-based or disk-based audio/video production sys- 
tems. Newsroom computer systems, such as those pre- 

10 viously available from BaSys, and now from Avid 
Technology under the name NetStation, have devel- 
oped from systems which were developed to receive 
news agency copy and provide simple word processing 
and communications facilities. In more recent years, 

15 add-ons of various kinds have been developed which 
provide some integration of the text production opera- 
tion with the audio/video production operation. How- 
ever, only limited integration of the text and audioMdeo 
data has been achieved, thereby providing only limited 

20 multimedia capability. 

[0008] In a typical news production organization, a 
journalist develops an idea for a story, and determines 
how various audio/video clips should be used in the 
story. Often, the journalist will preview audio/video foot- 

25 age that has been archived, and select portions of the 
archived footage, called clips, for use in the story. Then, 
the journalist provides instructions to an editor who 
edits the clips to produce a final form of the story that is 
suitable for broadcast. 

30 [0009] In some instances, particularly if the story is 
complex, the journalist may wish to prepare a rough 
form of the story and provide the rough form to the edi- 
tor for final preparation. A rough form of what the jour- 
nalist expects for the final form of the story is better than 

35 verbal or hand written instructions. To this end, if the 
journalist wishes to incorporate video from a previous 
broadcast that is contained in a video tape archive, the 
journalist must request that the tape be retrieved manu- 
ally, and must then review the tape in an edit bay or a 

40 similar location. The journalist may then perform some 
preliminary editing of the archived video, with other 
material such as video of recent events, text and graph- 
ics received over news wire services, and archived text, 
before providing the rough form to the editor and 

45 instructing the editor to prepare the final form of the 
story for broadcast. In present day systems, the capabil- 
ity to perform the above-identified functions is not avail- 
able to the journalist in a newsroom system, but as 
discussed above, must be performed remotely, for 

so example, in an edit bay. 

[0010] Furthermore, a journalist may wish to pre- 
pare a story about a particular event while the event 
unfolds. If the journalist has access to a live feed of the 
event, it is likely that the journalist will record the event 

55 on a video tape using a video tape recorder (VTR), or in 
a file on a disk using a non-linear disk-based 
audio/video production system. If the journalist is 
recording the event on video tape and wishes to prepare 
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a rough form of the story by integrating recorded por- 
tions of event, the journalist must stop the VTR, and 
rewind the video tape to the specific recorded portions 
intended for integration. If new developments occur 
while the journalist is using the VTR to integrate the 5 
recorded portions, the live feed of these new develop- 
ments will be lost unless the live feed is recorded simul- 
taneously on a second tape using a second VTR. 
Similarly, if the journalist is using a conventional non-lin- 
ear disk-based audio/video production system to record w 
the live feed in a file, the journalist must terminate the 
recording before the journalist can access the recorded 
portions from the file for integration into the story. To 
record additional developments of the event on the disk- 
based system, the journalist must record the additional 75 
developments into a second file. Storage of the event 
among multiple tapes and files is inefficient and requires 
additional overhead to keep track of multiple tapes and 
files. 

20 

Summary of the Invention 

[0011] An embodiment of the invention is directed 
to a multimedia system that includes a multimedia cap- 
ture and encoding system that captures multimedia 25 
data, and provides a first compressed version of the 
multimedia data having a first resolution and a second 
compressed version of the multimedia data having a 
second resolution that is different from the first resolu- 
tion. The multimedia system further includes a multime- 30 
dia storage system, coupled to the multimedia capture 
and encoding system, that stores multimedia informa- 
tion including the first and second compressed versions 
of the multimedia data. The multimedia system further 
includes a video editing and playback system coupled to 35 
the multimedia storage system. The video editing and 
playback system includes editing circuitry that gener- 
ates a composition that uses a portion of the first com- 
pressed version, and playback circuitry that plays the 
composition using a portion of the second compressed 40 
version that corresponds to the portion of the first com- 
pressed version. 

[0012] Another embodiment of the invention is 
directed to a multimedia system that includes a multi- 
media capture and encoding system that captures mul- 45 
timedia data, and provides a compressed version of the 
multimedia data having a first resolution. The multime- 
dia system further includes a multimedia storage sys- 
tem, coupled to the multimedia capture and encoding 
system that stores multimedia information including the so 
compressed version of the multimedia data, and pro- 
vides to a network the compressed version of the multi- 
media data substantially simultaneously as the 
compressed version is stored. 

[0013] According to an embodiment of the inven- 55 
tion, the multimedia storage system includes a server 
• coupled to-the network that sends the compressed ver- 
sion on the network. 
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[0014] According to s^ffher embodiment, the multi- 
media system further includes a video host coupled to 
the network that sends a first request to the server for a 
first portion of the compressed version of the multimedia 
data, determines an amount of time to wait based on a 
length of the first portion and a response time of the first 
request, and sends a second request to the server for a 
second portion of the compressed version of the multi- 
media data after waiting the determined amount of time. 

Brief Description of the Drawings 

[0015] For a better understanding of the present 
invention, reference is made to the accompanying draw- 
ings which are incorporated herein by reference and in . 
which: 

FIG. 1 is a block diagram illustrating components of 
a typical television news operation; 
FIG. 2 is a block diagram illustrating components of 
a typical television news operation having 
audio/video production capabilities integrated with 
on-air operations; 

FIG. 3 is a block diagram of a digital multimedia 
system according to an embodiment of the present 
invention; 

FIG. 4 is a block diagram of a digital multimedia 
system having a capture manager and an asset 
manager according to an embodiment of the 
present invention; 

FIG. 5 is a block diagram of a digital multimedia 
system having multiple low resolution encoders and 
multiple high resolution encoders according to an 
embodiment of the present invention; 
FIG. 6 is a block diagram of a digital multimedia 
system having a browse server according to an 
embodiment of the present invention; 
FIG. 7 is a flow diagram of a method performed by 
a video host of a digital multimedia system, accord- 
ing to an embodiment of the present invention; 
FIG. 8 is a flow diagram of a method performed by 
a browse server of a digital multimedia system, 
according to an embodiment of the invention; 
FIG. 9 is a view of a dialog window of a capture 
manager of a digital multimedia system according 
to an embodiment of the present invention; 
FIG. 10 is a block diagram of a digital multimedia 
system having a core newsroom system, a multi- 
media archive, and a video production system, 
according to an embodiment of the present inven- 
tion; 

FIG. 1 1 is a view of a graphics user interface of a 
digital multimedia newsroom production system 
according to an embodiment of the present inven- 
tion; and 

FIG. 12 is a diagram of a multimedia file structure 
according to an embodiment of the present inven- 
tion. 
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Detailed Description 



[0016] Fig. 3 shows a digital multimedia system 50 
for managing motion video data in accordance with an 
embodiment of the invention. The multimedia system 50 5 
enables one or more users to manipulate effectively 
motion video data, text, graphics and audio (i.e., multi- 
media data) and generate a multimedia composition. In 
particular, the system 50 substantially simultaneously 
encodes a low resolution version and a high resolution 10 
version of multimedia data. A journalist using the sys- 
tem generates a composition using a portion of the low 
resolution version, and an editor plays the composition 
using a portion of the high resolution version that corre- 
sponds to the portion of the low resolution version. is 
[0017] The multimedia system 50 includes a multi- 
media capture and encoding system 52 that captures 
multimedia data, and substantially simultaneously pro- 
vides the first compressed version of the multimedia 
data having the first resolution, and the second com- 20 
pressed version of the multimedia data having the sec- 
ond resolution that is different than the first resolution. 
[0018] The multimedia system further includes a 
multimedia storage system 54, coupled to the multime- 
dia capture and encoding system 52, that stores multi- 25 
media information including the first and second 
compressed versions of the multimedia data. In particu- 
lar, the multimedia storage system 54 includes a digital 
computer-readable and writable non-volatile random- 
access medium, such as a magnetic disk, for storing the 30 
first and second compressed versions digitally and non- 
linearly. 

[0019] The multimedia system 50 further includes a 
video editing and playback system 56 coupled to the 
multimedia storage system 54. The video editing and 35 
playback system 56 includes editing circuitry 58 that 
generates a composition that uses a portion of the first 
compressed version, and playback circuitry 60 that 
plays the composition using a portion of the second 
compressed version that corresponds to the portion of 40 
the first compressed version. The composition includes 
one or more data structures that define a list of video 
entries. Each video entry indicates a name of a file con- 
taining video information, and a range of the file that 
defines a portion of the video information. The editing 45 
circuitry 58 and the playback circuitry 60 are typically 
used by a journalist and an editor, respectively. Alterna- 
tively, both the editing circuitry 58 and the playback cir- 
cuitry 60 may reside on a single graphics workstation. 
[0020] Another embodiment of the invention is a so 
newsroom production system 700 which is illustrated in 
Fig. 4. The system 700 substantially simultaneously 
encodes a low resolution version and a high resolution 
version of multimedia data, and enables a journalist 
using the system to generate a composition using a por- 55 
tion of the low resolution version, and an editor to play 
the composition using a portion of the high resolution 
version that corresponds to the portion of the low reso- 



lution version. As^Whe embodiment of Fig. 3, the sys- 
tem 700 enables a news production organization to 
manipulate effectively multimedia data including motion 
video clips from a variety of sources, as well as text, live 
presentations by announcers and associated graphics. 
[0021 ] The system 700 is similar to the system 50 
described above in that the system 700 includes a mul- 
timedia capture and encoding system 710, a multimedia 
storage system 730 and a video editing and playback 
system 750. The system further includes a first compu- 
ter network 704 and a second computer network 706 
that are coupled to a bridge 708. Each of the multimedia 
capture and encoding system 710, the multimedia stor- 
age system 730, and the video editing and playback 
system 750 is coupled to the first network 704 and the 
second network 706. According to an embodiment of 
the invention, the network 706 is an ATM network such 
as Avid Net available from Avid Technology, Inc., Tewks- 
bury, Massachusetts, which is described in U.S. Patent 
Application No. 08/215,849, which is hereby incorpo- 
rated by reference. The system 700 also includes an 
input 702 for receiving multimedia data, from one or 
more sources. 

[0022] The multimedia capture and encoding sys- 
tem 71 0 includes a first encoder 712 coupled to the first 
network 704, a second encoder 71 6 coupled to the sec- 
ond network 706, and an encoding controller 714 inter- 
connected between the encoders 712 and 716. The 
encoding controller 714 is also referred to as a capture 
manager. Each of the encoders 712 and 716 is further 
coupled to the video input 702 to receive the multimedia 
data. 

[0023] The multimedia storage system 730 includes 
a first video server 732 coupled to the first network 704, 
a second video server 736 coupled to the second net- 
work 706, and an asset manager 734. The asset man- 
ager 734 is coupled to each of the second video server 
736, the capture manager 714 and the second encoder 
716. 

[0024] The video editing and playback system 750 
includes a first workstation 742 coupled to the first 
graphics network 704, and a second graphics worksta- 
tion 744 coupled to the second network 706. The first 
graphics workstation includes first editing circuitry 752 
coupled to the first network 704. The second graphics 
workstation includes playback circuitry 754 coupled to 
the second network 706, and second editing circuitry 
756 coupled to the second network 706. Alternatively, 
the playback circuitry 754 and the second editing cir- 
cuitry 756 may reside on separate graphics worksta- 
tions each of which is coupled to the second network 
706. Both the playback circuitry 754 and the second 
editing circuitry 756 are further coupled to the asset 
manager 734. 

[0025] When the system 700 is in operation, the 
first and second encoders 712 and 716 substantially 
simultaneously receive a multimedia data signal from 
the input 702. The first encoder 712 outputs over the 
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network 704 a signal containing a first compressed ver- 
sion of the multimedia data. The second encoder 716 
outputs over the network 706 a signal containing a sec- 
ond compressed version of the multimedia data. The 
resolution of the first compressed version is different 5 
than the resolution of the second compressed version. 
In one embodiment, the first and second resolutions dif- 
fer from a time perspective so that one of the versions 
uses less frames than the other over a given interval of 
time. In another embodiment, the first and second reso- to 
lutions differ spatially, i.e., in the number of pixels used 
to represent a still image, so that one of the versions 
provides images of a better clarity than the other ver- 
sion. In yet another embodiment, the first and second 
resolutions differ both temporally, i.e., the number of 
images per second of motion video, and spatially. In a 
particular embodiment of the invention, the first com- 
pressed version is an MPEG-1 (ISO/IEC 11172-1 
through 9) encoded stream, and the second com- 
pressed version is a 60 field per second motion- J PEG 
(MJPEG) encoded stream of broadcast television qual- 
ity images so that the first and second compressed ver- 
sions have different temporal and spatial resolutions. 
[0026] The first video server 732 receives and 
stores the first compressed version from the first 
encoder 712. The second video server 736 receives 
and stores the second compressed version from the 
second encoder 716. Storage of the first and second 
compressed versions occurs substantially simultane- 
ously. In a preferred embodiment, the first video server 
732 is a low resolution video server that stores low res- 
olution multimedia data such as Avid BrowseServer, 
and the second video server 736 is a high resolution 
video server that stores high resolution multimedia data 
such as Avid MediaServer. Both Avid MediaServer and 
Avid BrowseServer are motion video storage devices 
available from Avid Technology, Inc., Tewksbury, Massa- 
chusetts. 

[0027] The capture manager 714 controls the asset 
manager 734 so that a correspondence between the 
first and second compressed versions is generated. In 
particular, the asset manager 734 initially creates and 
then maintains a mapping of the first and second com- 
pressed versions. In one embodiment, the mapping is 
achieved by storing file identification information and 
timecode data in a file. If a filename and timecode range 
identifying a portion of the first compressed version is 
provided to the asset manager 734, the asset manager 
can identify a portion of the second compressed version 
that corresponds to the portion of the first compressed so 
version. In particular, the asset manager 734 searches 
the file and retrieves a filename and a timecode range 
identifying the portion of the second compressed ver- 
sion that corresponds to the portion of the first com- 
pressed version. Accordingly, correspondence between 55 
the first and second compressed versions is achieved. 
[0028] Such a mapping mechanism may be imple- 
mented by using a form of dynamic linking as disclosed 
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in U.S. Patent ApplicatkJfWK 5,267,351 to Reber et al., 
which is hereby incorporated by reference. In particular, 
the asset manager 734 may maintain indications of high 
resolution video files which are equivalent to low resolu- 
tion files. This equivalency can be used to automatically 
and dynamically associate the appropriate high resolu- 
tion files with the low resolution files used by the journal- 
ist to create a multimedia composition. 
[0029] If timecode information is unavailable on the 
audio/video feed received by the input 702, the capture 
manager 714 ensures that timecode information is 
included in the encoded multimedia data. In particular, if 
timecode information is not included, either the capture 
manager 71 4 or the asset manager 734 adds timecode 
information to the second compressed version before it 
is stored in the second video server 736. 
[0030] The operation of the system 700 will now be 
described in connection with a newsroom setting. A 
journalist in the newsroom operates the first workstation 
742 and an editor operates the second workstation 744. 
The journalist generates a composition that uses a por- 
tion of the first compressed version of the multimedia 
data having the first resolution. It is not necessary that 
the composition be suitable for broadcast. Rather, the 
composition may be a rough form of the journalist's 
story that an editor can convert into a final form for 
broadcast. In particular, the journalist sends the repre- 
sentation of the composition (not including the media 
data) to the editor across the first and second networks 
704 and 706 through the bridge 708. When the editor 
receives the composition, the editor can play the com- 
position on the second workstation 744. When the sec- 
ond workstation 744 plays the composition, the second 
workstation plays a portion of the second compressed 
version of the multimedia data having the second reso- 
lution rather than the portion of the first compressed ver- 
sion used by the journalist. The editor converts the 
composition into the final broadcast form by performing 
editing operations, such as adding blend and fade tran- 
sitions between audio/video portions and other special 
effects, using the second editing circuitry 756. 
[0031] It should be understood that when the jour- 
nalist generates the composition using the first worksta- 
tion 742, the first compressed version of the multimedia 
data is transferred only through the first network 704. 
Similarly, when the editor plays the composition using 
the second workstation 744, the second compressed 
version of the multimedia data is transferred only 
through the second network 706. 
[0032] Also, it should be understood that, using the 
system 700, neither the journalist nor the editor leaves 
their respective workstations to retrieve audioArideo 
footage for integration into the composition. The journal- 
ist has access to the first compressed version stored in 
the first video server 732. Similarly, the editor has 
access to the second compressed version stored in the 
second video server 736. 

[0033] Another embodiments! the invention is 
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directed to a newsroom ^pTction system 800 which is 
illustrated in Fig. 5. This embodiment is similar to the 
embodiment of Fig. 4 except that it includes multiple first 
encoders 812, multiple second encoders 816, and mul- 
tiple first workstations 842. As in the other embodiments 
of the invention previously described, a news production 
organization can utilize effectively the system 800 to 
generate news stories that include broadcast quality 
motion video clips from a variety of sources. The encod- 
ers 812 are low resolution encoders that simultaneously 
output low resolution compressed versions of multime- 
dia data from various sources. The encoders 816 are 
high resolution encoders that simultaneously output 
high resolution compressed versions of the multimedia 
data from the various sources. 

[0034] A benefit of the system 800 is that the multi- 
media data transferred through the first network 804 is 
low resolution data which requires less bandwidth than 
high resolution data. Accordingly, several workstations 
842 can be connected to the first network .804 without 
experiencing substantial degradation in performance of 
the first network 804. The multimedia data transferred 
through the second network 806 is high resolution data 
which uses more network bandwidth than low resolution 
data. Nevertheless, the second network 806 provides 
suitable performance since it is isolated from network 
traffic caused by the first workstations 842. Accordingly, 
more than one second workstation 844 may be con- 
nected to the second network 806. The bridge 808 
allows certain signals to pass from one network to the 
other. In particular, the bridge 808 allows a journalist 
working on one of the first workstations 842 to send a 
generated composition, i.e., one or more data struc- 
tures that define a story, to an editor working on one of 
the second workstations 844. 

[0035] Since the system 800 includes more than 
one low resolution encoder 81 2 and more than one high 
resolution encoder 816, as illustrated in Fig. 5, the sys- 
tem 800 can capture, encode and store both low resolu- 
tion and high resolution versions of more than one 
audio/video feed simultaneously. The capture manager 
814 maintains control of the multiple encoding sessions 
simultaneously. For example, the input 802 may have a 
first terminal that is connected to a satellite feed so that 
the satellite feed can be encoded and stored by the sys- 
tem 800. The input 802 may have a second terminal that 
is connected to a live camera so that the camera feed 
can be encoded and stored simultaneously by the sys- 
tem 800. 

[0036] According to embodiments of the invention, 
a user can view and edit an encoded version of an 
audio/video feed while the encoded version is being 
encoded and stored in a file on one of the video servers. 
In accordance with these embodiments, a system 890 
includes an encoder, a browse server, and a worksta- 
tion, as shown in Fig. 6. These devices can be a portion 
of the multicast system 800 of Fig. 5. 
[0037] The first video server 832 illustrated in Fig. 5 



is suitable as'tnWProwse server B32 in Fig. 6. The 
browse server 832 includes a buffer cache 870 and 
disk-based memory 880, as shown in FIG. 6. As the 
browse server 832 receives an encoded version of an 
5 audio/video feed from the low resolution video encoder 
812, portions of the encoded version are initially cached 
in the buffer cache 870 by the browse server's operating 
system. The operating system writes, i.e., flushes these 
portions from the buffer cache 870 to a file on the disk- 
to based memory 880. As the operating system writes the 
portions to the memory, the operating system simulta- 
neously sends network packets including these portions 
onto the network 804 to one or more workstations 854 
for viewing and possible editing if a request for the 
rs encoded version is received from the one or more work- 
stations 854. 

[0038] In one embodiment of the invention, the 
browse server 832 uses the Windows NT operating sys- 
tem available from Microsoft, Corporation, Redmond, 

20 Washington, which permits data to be stored into a file, 
and simultaneously read from the file without file con- 
tention problems. The system 890 utilizes file access 
operations provided by the Windows NT operating sys- 
tem so that multimedia data can be flushed from buffer 

25 cache 870 to the disk-based memory 880, and simulta- 
neously sent to one or more workstations 842 through 
the network 804. In particular, according to an embodi- 
ment of the invention, the system utilizes a feature of the 
NT file system providing the ability to read data from a 

30 file while data is appended to the file. 

[0039] In one embodiment, the portions are multi- 
cast (or "pushed") over the network 804, and any work- 
stations 842 wishing to have access to the portions 
simply register a request with the browse server 832. 

35 Then, the browse server multicasts the portions to the 
workstations 842 over the network 804 using IP multi- 
casting as the browse server simultaneously stores the 
portions. 

[0040] In another embodiment, the browse server 

40 832 responds specifically to individual requests for 
encoded portions, i.e., the portions are "pulled" from the 
browse server 832 to the workstation 842. In this 
embodiment, the browse server 832 functions as a web 
server by providing packets of information onto a com- 

45 puter network in response to individual requests from 
various workstations 842. In particular, when a worksta- 
tion 842 wishes to receive a portion of the encoded ver- 
sion of multimedia data, the workstation 842 sends a 
request onto the network 804 that is received by the 

so browse server 832. The browse server 832 responds by 
sending a network packet containing a portion of the 
encoded version back to the workstation 842 substan- 
tially simultaneously as the encoded version is stored 
into a file on the browse server 832. 

55 [0041] In accordance with an embodiment of the 
invention, the workstation 842 performs the method 
1000 illustrated in* Fig. 7. In step 1002, the workstation 
842 sends a request to the browse server 832 for one or 
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more portions of the encoded version that is being 
simultaneously stored in the browse server 832. In step 
1 004, the workstation 842 waits until it receives portions 
of the encoded version from the browse server 832 in 
response to the request. In step 1006, the workstation 
842 receives and plays one or more portions, and deter- 
mines when to send a next request for more portions. 
The time for sending a next request depends on both 
the amount of video data received, e.g., the number of 
portions, and the time it took between sending the 
request and receiving the data. In step 1006, the work- 
station sends the next request expecting to receive one 
or more new portions of the encoded version a prede- 
termined amount of time before the workstation 842 is 
through playing the earlier received portions. Accord- 
ingly, the workstation 842 attempts to maintain some 
predetermined amount of lead time. In one embodi- 
ment, this lead time is approximately 0.5 seconds so 
that the workstation 842 sends the next request expect- 
ing that the next portions will be received 0.5 seconds 
before the previous portion is through playing. In step 
1 008, the workstation 842 checks whether the end of 
the file that stores the encoded version has been 
reached. If so, the method 1000 terminates. Otherwise, 
the workstation 842 repeats the method 1000. 
[0042] In accordance with an embodiment of the 
invention, the workstation 842 uses an active reader 
thread to acquire the new portions. If more than 6 sec- 
onds worth of material is stored by the workstation 842, 
the reader thread sleeps for a predetermined amount of 
time or until it is activated. 

[0043] In accordance with an embodiment of the 
invention, the browse server 832 performs the method 
1100 illustrated in Fig. 8. In step 1102, the browse 
server 832 opens a file to store the encoded version of 
multimedia data. In step 1104, the browse server 832 
polls the network 804 for requests for portions of the 
encoded version, and when a request is received from 
the workstation 842, the browse server 832 sends one 
or more portions of the encoded version to the worksta- 
tion 842. The browse server 832 can track which por- 
tions of the encoded version have been sent to the 
workstation 842 and which portions to send in response 
to the next request. In particular, if the workstation 842 
includes an identification ID with its request, the browse 
server 832 can use the ID to find determine which file 
and which read block need to be accessed, and then 
send the read block and other information such as time- 
code and length information of the portion or portions of 
the encoded version defined by the read block. Alterna- 
tively, the workstation 842 tracks which portions of the 
encoded version are needed next, and sends an indica- 
tion of which portions it needs with the next request In 
step 1 1 06, the browse server 832 determines whether 
the encoded version has been completely stored, e.g., 
whether the encoder 812 has been stopped. If so, the 
browse server 1 106 proceeds to step 1 108 and closes 
the file, and sends an end of file indication along with 
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any remaining unsent pSWrohs when a next request is 
received from the workstation 842. Otherwise, the 
browse server 832 proceeds to step 1104 to continue 
polling the network and storing the encoded version in _ 
the file. 

[0044] It should be understood that the workstation 
842 may be an Internet client by having an IP address, 
and the browser server 832 is effectively a server, such 
as an http server or other kind of server that uses the 
TCP/IP protocol. According to a preferred embodiment, 
communications between the workstation 842 and the 
browse server 832 are "connectionless." That is, the 
requests sent from the workstation 842 to the browse 
server 832 establish a connection only for the period of 
time required to transmit network packets of the 
request. Similarly, another connection is established 
between the browse server 832 and the workstation 842 
for transfer of one or more portions of the encoded ver- 
sion across the network 804. Otherwise, no connection 
exists, i.e., no connection stream remains open. 
[0045] In one embodiment, http server software 
may be used by the browse server 832 to handle 
responses from the workstations 842 which are config- 
ured as web hosts. Such software is Microsoft Internet 
Information Server, or Microsoft Peer Web Services, 
available from Microsoft, Corporation. Redmond, Wash- 
ington. 

[0046] Using either of the foregoing embodiments, 
a journalist working at the workstation 842 may view 
and edit an encoded version of the multimedia data 
while it is being stored in the browse server 832. 
Accordingly, the journalist may prepare a composition 
that includes portions of an encoded version of multime- 
dia data, while the multimedia data is being simultane- 
ously stored in a browse server. Furthermore, the 
journalist is not burdened with having to store the 
encoded version in multiple files on the same browse 
server 832. Portions of the encoded version on the 
workstations 842 can be accessed with a maximum of 5 
seconds of delay from the time the audioA/ideo feed is 
first provided to the input 802. 

[0047] Each journalist may create a recording ses- 
sion and maintain control of the recording session using 
a graphical user interface of the capture manager 814. 
This user interface also includes some access features 
(e.g.. viewing and editing capabilities described above) 
allowing the journalist to access portions of an encoded 
version as the encoded version is being simultaneously 
recorded and stored. An example of the graphical user 
interface is illustrated in FIG. 9. The interface is in the 
form of a dialog window 900 that includes one or more 
property page displaying one or more respective encod- 
ing configurations. Each property pages includes but- 
tons that enable the user to send commands and 
information to the capture manager 814 using conven- 
tional input methods with a mouse and a keyboard. 
Other conventional input mechanisms can be substi- 
tuted for the mouse and keyboard. In the particular 
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example shown in FIG. ^^^dialog window 900 has six 
property pages named: Network Feedl, Network 
Feed2, Archive VTR, Projects VTR, Satellite Feedl, 
and Satellite Feed 2. The property page for Network 
Feedl is shown as being presently in the foreground by 
"Network Feedl " being displayed as the source 902. 
The other property pages are shown in the background 
by tables 940 with their respective names. In this partic- 
ular example, the high resolution encoder 81 6 is named 
"Jupiter" and the low resolution encoder is named 
"MR1". The bottom area 904 of the dialog window 900 
displays a plurality of buttons including an -Exit" button 
906 for exiting the graphical user interface of the capture 
manager 814, a "New" button 908 for creating a new 
property page for a new encoding configuration, a 
"Delete" button 910 for deleting a property page, and a 
"Help" button 912 for obtaining help through a help win- 
dow (not shown). 

[0048] The dialog window 900 further displays 
recording status information including an eJapsed time 
914 of the encoding session, a start time 916 that is 
assigned to the encoded version of the multimedia data 
being stored, a recorded headframe 91 8 that is used as 
a graphical image representation of the encoded ver- 
sion, and a flashing status 920 that indicates a current 
state of the encoding session. 

[0049] The dialog window 900 further displays addi- 
tional control buttons depending on the configuration of 
the encoding session as identified by its property page. 
For example, as shown in Fig. 9, the Network Feedl 
property page includes encoder control buttons 922: 
"Standby" 924, "Start" 926 and "Stop" 928, that allow 
the user to respectively pause, start and stop an encod- 
ing session. The Network Feedl property page further 
includes a "Previewer" (not shown) that allows the user 
to view progress of the encoding session, a "Synchro- 
nizer 932 that allows the user to advance to the end of 
the currently encoded video to view the latest results of 
the encoding session, a "Metadata Edit Controller" 930 
that allows the user to view and modify portions of the 
encoded version, and a "Headframe Grabber* 934 that 
allows the user to select, as the headframe for the 
encoded version, any frame in the encoded version that 
has been stored. 

[0050] Some of the operations of the capture man- 
ager B14 will described in further detail. Each of the 
property pages in the dialog window 900 is tabbed, as 
shown in the area 940 of Fig. 9. When the user selects 
one of the tabs, the capture manager 814 displays the 
property page associated with the selected tab in the 
foreground of the dialog window 900. If the user cannot 
find an appropriate configuration to select and deter- 
mines that a new configuration is needed, the user may 
create a new configuration and a new property page 
associated with the new configuration by pressing the 
"New" button 908. The capture manager 814 will 
respond by prompting the user for information regarding 
the new configuration until it has enough information to 
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begin a new encWffhg session. The capture manager 
814 begins encoding when the user selects the "Start" 
button 926. In particular, the capture manager 814 
sends a signal to the low resolution encoder 812 

5 through connection 820 (see FIG. 5) causing it to begin 
encoding. Alternatively, the capture manager 814 sends 
this signal to the low resolution encoder 812 when the 
capture manager 814 receives a signal from the high 
resolution encoder 816 through connection 818 indicat- 

10 ing that the high resolution encoder 816 has started 
encoding. Accordingly, if the user has started the high 
resolution encoder 81 8, the low resolution encoder 81 2 
is started automatically and simultaneously. 
[0051] It should be understood that the user inter- 

75 face enable a journalist to control multiple live feeds 
simulaneously from one graphics workstation. When an 
operation is desired for one of the encoding sessions, 
the journalist brings the property page for that encoding 
session to the foreground in the dialog window 900 and 

20 performs the desired operations. Then, the journalist 
can perform an operation on a different encoding ses- 
sion by bring it to the foreground. 
[0052] The journalist using the capture manager's 
dialog window 900 can view any portion of the encoded 

25 version as long as it has been stored in a file in the 
browse server 832. In particular, the journalist may jump 
to the beginning of the version, jump to the middle of the 
version, and jump to the end of the version. All of these 
access methods can occur while browse server 832 

30 continues storing additional portions of the encoded 
version in the same file. 

[0053] Furthemore, the journalist may add markers 
to the portions of the encoded version in real time. The 
journalist is not required to wait until an encoding ses- 
35 sion is over before viewing and marking multimedia 
data. 

[0054] Another embodiment of the invention is 
directed to a multimedia newsroom production system 
90, as illustrated in Fig. 10. This system is described in 
40 U.S. Patent Application Serial No. 08/631,441, filed on 
April 12, 1996, which is hereby incorporated by refer- 
ence. The newsroom production system 90 enables a 
news production organization to manipulate effectively 
multimedia data to generate news stories for broadcast- 
's ing. Each generated news story may include several 
broadcast quality motion video clips from various 
sources. The system 90 includes three major systems, 
a core newsroom system 100, a multimedia archive 
200, and a video production system 300. 
so [0055] In one embodiment, the components of the 
systems are interconnected through a single digital net- 
work. Preferably, the single digital network is a 1 00 Mb/s 
network. 

[0056] In another embodiment, the components of 
55 the core newsroom system and the multimedia archive 
are interconnected using a first digital network 400, and 
the components .of the video production system are 
interconnected with a second digital network 410. An 
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adaptor box 420 is connected to both the first digital net- 
work 400 and the second digital networks 41 0 to enable 
communication between the two networks. In a pre- 
ferred embodiment of the invention, the first digital net- 
work 400 is implemented using an Ethernet system 5 
having a data rate equal to, or greater than, 1 00 Mb/s, 
and the second digital network 410 is implemented 
using an Ethernet system having a data rate equal to, or 
greater than, 10 Mb/s. The adaptor box 420 may be 
implemented using one of a number of commercially w 
available products such as a FastNet 10 available from 
Cabletron Systems, Inc, Rochester, New Hampshire. 
[0057] Each of the major components of the news- 
room production system 90 is described in greater detail 
below. 15 

VIDEO PRODUCTION SYSTEM 300 

[0058] The video production system 300 provides 
audio/video capture, media data editing, and manage- 20 
ment and control of high quality multimedia data suita- 
ble for broadcast. The multimedia data can be any form 
of information that can be represented in a digital form. 
The video production system includes a digital playback 
system 31 0, a video editor 320, a media recorder 330 25 
connected to an MPEG encoder 340, a media server 
350 including an asset manager 360, a high bandwidth 
data network 364, and a graphics workstation 370. 
[0059] The media server 350 is a large scale com- 
puter that stores and delivers high quality audio and 30 
motion JPEG video (MJPEG), suitable for broadcast, in 
conjunction with the other devices of the video produc- 
tion system 300. The media server 350 can also func- 
tion as an archive system for multimedia data produced 
in the video production system 300. In a preferred 35 
embodiment of the invention, additional near-line stor- 
age and off-line storage is provided on a digital data 
storage medium, such as tape or optical disks, to relieve 
the media server of archive responsibilities to provide 
additional on-line storage capabilities within the media 40 
server 350. 

[0060] An asset manager 360 is an integral part of 
the media server 350 and is implemented as software in 
the media server 350. The asset manager 360 stores 
information and is the tool used to manage the data 45 
stored in the near-line storage and the off-line storage. 
The material stored in the media archive can be auto- 
matically moved to on-line status on the media server by 
the asset manager 360. The asset manager 360 con- 
tains search support data for locating media objects so 
stored in the media server 350, in the near-line storage 
system and in the off-line storage system. The asset 
manager 360 also contains composition information 
that can be used to capture, edit, and play back the 
media objects stored in the media server 350. As 55 
described below in greater. detail, the media server 350 
also provides translation of low resolution media data 
compositions, generated within the core newsroom sys- 
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tern, to high resolution me^raata compositions for edit- 
ing and playback within the video production system. In 
a preferred embodiment, the media server 350 is imple- 
mented using an Avid MediaServer™ available from 
Avid Technology. Inc., Tewksbury, Massachusetts. 
[0061] The media recorder 330 is a disk-based dig- 
ital recording workstation which is used to capture 
audio/video data and provide digitization and compres- 
sion of the audio/video data. The media recorder 330 
digitizes, compresses and records audio/video material 
and transmits the digitized compressed data to the 
media server over the high speed network for storage 
on the media server 350. 

[0062] In a preferred embodiment of the invention, 
the media recorder 330 uses an MJPEG encoding 
scheme to generate high quality, high resolution, com- 
pressed digital data suitable for broadcast. In the pre- 
ferred embodiment, an MPEG encoder 340 is coupled 
to the media recorder 330 to also provide MPEG com- 
pression capability. As described in greater detail below, 
the addition of the MPEG encoder 340 to the media 
recorder 330 provides the system with a dual-digitizing 
capability for media data recorded by the media 
recorder 330. The MPEG encoder provides greater 
compression of the data than the media recorder 330, 
thereby allowing the data to be efficiently transmitted 
over the Ethernet network 400 to be played on the jour- 
nalist workstations 110. As shown in Fig. 4 the MPEG 
encoder 340 has a direct connection to the digital net- 
work 400 to provide MPEG encoded media data to the 
multimedia archive 200. 

[0063] In a preferred embodiment, the media 
recorder 330 is implemented using an Avid Media 
Recorder™ available from Avid Technology Inc., Tewks- 
bury, Massachusetts. 

[0064] The video editor 320 is a full-feature, digital, 
non-linear video editing workstation specifically tailored 
to provide functions for news editing. The video editor 
provides editing of high resolution broadcast quality 
images provided by the media server 350. In a preferred 
embodiment, the video editor is implemented using a an 
Avid NewsCutter™ or an Avid Media Composer®, both 
of which are available from Avid Technology Inc., 
Tewksbury, Massachusetts. 

[0065] The digital playback system 31 0 is a digital, 
disk-based playback system that manages the broad- 
cast to air of multimedia data produced and stored 
within the video production system 300. The digital play- 
back system 31 0 plays materials stored either locally or 
on the media server 350 in accordance with play lists 
generated from a program lineup created on one of the 
journalist workstations 110 within the core newsroom 
system 1 00, or on a workstation directly coupled to the 
video production system (not shown). In a preferred 
embodiment of the invention, the digital playback sys- 
tem 31 0 is implemented using an Avid AirPlay® availa- 
ble from Avid Technology, . Inc., Tewksbury, 
Massachusetts. 
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[0066] The high ba^pfth network 364 provides 
high speed communication between the components of 
the video production system 300. In a preferred embod- 
iment of the invention, the high bandwidth network 364 
is rnplemented using an ATM network as described in 5 
co-pending U.S. Patent Application 08/249,849, titled 
An Apparatus and Computer Implemented Process For 
Providing Real-Time Multimedia Data Transport in a 
Distributed Computing System, which is incorporated 
herein by reference. The high bandwidth network 364 10 
supports real time playback of broadcast quality 
MJPEG video and multi-track audio over fiberoptic net- 
works. 

[0067] The graphics workstation 370 is used for 
generating and editing graphics material for broadcast is 
from and storage in the video production system. In a 
preferred embodiment, the graphics workstation 370 is 
implemented using a Matador Workstation available 
from Avid Technology, Inc., Tewksbury, Massachusetts. 
[0068] It should be understood that , the media 20 
recorder 330 and the MPEG encoder 340 form a multi- 
media capture and encoding system, as illustrated in 
the embodiment of Fig. 3. In particular, the combination 
of the media recorder 330 and the MPEG encoder 340 
captures multimedia data, and substantially simultane- 25 
ously provides a first compressed version of the multi- 
media data having a first resolution (e.g., MPEG), and a 
second compressed version of the multimedia data hav- 
ing a second resolution (e.g., MJPEG) that is different 
than the first resolution. 30 
[0069] It should be further understood that the 
graphics workstation 370 forms playback circuitry 60 of 
a video editing and playback system 56, as illustrated in 
Fig. 3. In particular, the graphics workstation plays com- 
positions that use compressed versions of multimedia 35 
data stored in the media server 350. As will be 
described below, the compositions may be generated 
by the core newsroom system 100 using different com- 
pressed versions of multimedia data stored in the multi- 
media archive system 200. 40 

CORE NEWSROOM SYSTEM 100 

[0070] The core newsroom system 1 00 consists pri- 
marily of a number of journalist workstations 1 1 0 and a 45 
pair of news servers 120. Fig. 10 shows a newsroom 
system having three journalist workstations 110. In 
embodiments of the invention, the number of worksta- 
tions 1 10 actually used may be much greater than three, 
and the actual number of journalist workstations 110 so 
that may be used in the system is based on several fac- 
tors including the amount of network activity generated 
by each user of the workstations and by the amount of 
delay each user will tolerate in accessing the system. 
[0071] In a preferred embodiment of the invention, 55 
each of the journalist workstations 1 1 0 is implemented 
using an M PC III compliant workstation. 
[0072] The journalist workstation 110 provides 
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access to multinWBW data from a variety of sources and 
includes the tools (i.e. software) necessary to create a 
multimedia storyboard of a news story for broadcast. 
The multimedia data available to the journalist includes 
the low resolution MPEG video data captured by the 
media recorder. In one embodiment of the invention, 
each of the journalist workstations 1 1 0 includes a video 
port for receiving video from, for example, a VTR. Each 
of the journalist workstations 110 also includes a serial 
port for controlling the VTR. The graphics user interface 
of the journalist workstation 1 1 0 and the functions avail- 
able to a user of the journalist workstation 110 are 
described in greater detail below. 

[0073] The news server 120 provide management 
and storage of the multimedia data in the newsroom 
environment. The news servers 120 are configured as 
distributed processors with mirrored data bases to pro- 
vide maximum reliability and performance. Other cen- 
tralized functions, such as communications functions, 
are managed by the news servers 120. In a preferred 
embodiment, the news servers 120 are implemented 
using an Avid NewsServer available from Avid Technol- 
ogy, Inc., Tewksbury, Massachusetts. The news servers 
120 have external connections 122 for providing access 
to news wire services and to allow remote access to the 
news servers 120 from users external to the core news- 
room system. 

[0074] The core newsroom system 100 may also 
include one or more terminal servers 140 to provide 
connection to the digital network 400 for user terminals 
130. The user terminals may be one of several different 
terminals used in prior art systems primarily for text 
processing and communications functions. A device 
controller 150, or a number of device controllers 150, 
may also be coupled to the digital network 400 to pro- 
vide control of several multimedia devices, such as tele- 
prompters, from the journalist workstations. 
[0075] It should be understood that a journalist 
workstation 110 of the core newsroom system 100 in 
combination with a graphics workstation 370 of the 
video production system 300 form of a video editing and 
playback system 76, as illustrated in the embodiment of 
Fig. 3. The journalist workstation 110 forms editing cir- 
cuitry 58 that generates a composition that uses a por- 
tion of a first compressed version of multimedia data 
having a first resolution. As stated above, the graphics 
workstation 370 forms playback circuitry 60 that plays 
the composition using a portion of a second com- 
pressed version of the multimedia data stored in the 
media server 350. 

MULTIMEDIA ARCHIVE SYSTFM Pnn 

[0076] The multimedia archive (MMA) 200 includes 
a library server 21 0 and one or more object servers 220. 
The library server 21 0 holds catalog and search support 
meta data for locating objects stored in the multimedia 
archive 200. 
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[0077] The object server 220 provides the primary 
storage media for browsing and archival of materia! 
generated during news gathering and production proc- 
esses. The object server 220 works in conjunction with 
the library server 21 0 to facilitate distribution of multime- 
dia material to the journalist workstations 110. The 
objects stored in the multimedia archive can be low res- 
olution versions of video, audio, graphics, and text. The 
MMA can be used to store finished stories, audio, video 
and other content for reuse in creating new stories. In a 
preferred embodiment, the multimedia archive 200 is 
implemented using the IBM Digital Library 5765-258. 
[0078] It should be understood that the multimedia 
archive system 200 in combination with the media 
server 350 of the video production system form a multi- 
media storage system 54, as illustrated in the embodi- 
ment of Fig. 3. The multimedia archive system 200 and 
the media server 350 are coupled to the media recorder 
330 and the MPEG encoder 340 that form the multime- 
dia capture and encoding system 52, and are further 
coupled to the journalist workstations 110 and the 
graphics workstation 370 that form the video editing and 
playback system 56. The multimedia archive system 
200 and the media server 350 store multimedia informa- 
tion including the first and second compressed versions 
of the multimedia data, which are described above. 



f rHUs v 



10 



OPERATION OF THE NEWSROOM PRODUCTION 
SYSTEM 90 

[0079] The operation of the digital multimedia 
newsroom production system 90 shown in Fig. 1 0 is 
described below. The operation of the system 90 can be 
described as a collection of distinct function specific 
workloads characterized at a high level as asset crea- 
tion, asset use, asset storage, and asset administration. 
The system 90 provides the capability for the following 
functions: 

• News wire text capture, storage, and catalog; 

• News story text creation, storage, and catalog; 

• High resolution video capture, edit, playout, storage 
and catalog; 

• Video production system low resolution media data 
editing; 

• Real-time dual resolution digitization, storage and 
catalog; 

• Low resolution video browsing and editing; and 

• High-resolution playout and editing of low resolution 
composition 

Each of the functions described above, along with user 
interlaces for accomplishing these functions, are 
described below in greater detail. 

News Wire Text Capture. Storage and Catalog 

[0080] The news servers 120 provide capability for 
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capture and storage of rlWIS wire text data through the 
external interfaces 122. News wire text stories are cap- 
tured by the news servers 120 and cataloged in a data- 
base of the news servers 120. A user of one of the 
journalist workstations 110 may access the news* serv- 
ers* databases as a system librarian to search, browse 
and retrieve the wire service data stored in the data- 
bases of the news servers 1 1 0. It is not generally neces- 
sary to store all text stories captured by the news 
servers 1 1 0 in the multimedia archive 200. A system 
administrator may access the news servers through one 
of the journalist workstations 1 1 0, browse the catalog of 
data received from the news wires, determine what sto- 
ries are appropriate for storage in the multimedia 
archive 200 and command the news servers 120 to 
transfer selected data to the multimedia archive 200 for 
storage. 

News Story Text Creation. Storage, and Catalog 

[0081] A user of the journalist workstation 110 can 
access text through the news servers 120 and can cre- 
ate text and scripts from scratch or can use existing text 
and scripts stored in the news servers 1 20 or in the mul- 
timedia archive 200 in the creation of text and scripts. 
The user can search, browse and retrieve text data 
stored in the news servers 120 and the multimedia 
archive 200. The user can perform this searching and 
browsing using complex, full-text search techniques, 
thereby allowing efficient research by focusing the 
searching to retrieve data specifically relevant to the 
user's needs. 

High Resolution Video Capture. Edit Plavout. Storage 
and Catalog 

[0082] High resolution media data utilized by the 
video production system is captured in the system by 
the media recorder 330. The high resolution media data 
is captured in the media recorder 330, digitized and 
compressed using a broadcast quality compression 
technique such as M JPEG. The media data captured by 
the media recorder 330 is transferred in compressed 
form to the media server 350 and is registered and 
stored in the media server 350 by the asset manager 
360. As discussed further below, in a preferred embodi- 
ment of the invention, a low resolution version of the 
media data is simultaneously created with the high res- 
olution media data. 

[0083] The high resolution media data can be 
browsed and edited using the video editor 320 and can 
be broadcast to air using the digital playback system 
310. ' 

Video Production System Low Resolution Media Data 
Editing 

[0084] As discussed above, Low. resolution video is 
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used by the journalist wa^^tions 110 to provide lim- 
*ited editing capability. A user of the video production 
system 300, for example a user of the video editor 320, 
may wish to edit low resolution media data. The low res- 
olution media data may either be a low resolution com- 5 
position created by a user of a journalist workstation 
1 1 0 or a low resolution version of media data captured . 
by the media recorder 330. In either case, the video pro- 
duction system 300 user may search the multimedia 
archive 200 over the network 400 or may search the io 
asset manager 360 over the network 400 to retrieve the 
low resolution media data. After editing the low resolu- 
tion media data, the video editor 320 may transfer 
edited low resolution media data to the multimedia 
archive 200 for cataloging and storage therein. 15 

Real-time Dual Resolution Digitization. Storage and 
Catalog 

[0085] As described above, news video production 20 
from the journalist workstation 110 requires that an edit- 
able form of media data be available to a user of the 
journalist workstation 110. The low resolution media 
data is stored in, cataloged by and retrieved from the 
multimedia archive 200. The low resolution media data 25 
is captured in the system 90 using the media recorder 
330. The media recorder 330 performs a dual resolution 
digitization of media data to be captured by the system 
90. 

[0086] When media data is captured, the media 30 
recorder 330, in conjunction with the MPEG encoder 
340, performs a dual resolution digitization of the media 
data to simultaneously produce a high resolution ver- 
sion of the media data and a low resolution version of 
the media data. As discussed above, the high resolution 35 
version of the media data is digitized and compressed in 
a preferred embodiment using an MJPEG encoding for- 
mat. The low resolution video is compressed in a pre- 
ferred embodiment using known, high compression 
encoding techniques such as MPEG or Quick Time, 40 
available from Apple Computer, tnc, Cupertino, CA. 
Although it is preferred to use either MPEG or Quick 
Time, another compression technique which results in a 
high compression ratio of the media data may also be 
used. By performing simultaneous capture of both the 45 
high resolution version and the low resolution version of 
the media data, both forms of media data are immedi- 
ately available in the system 90 so that story editing can 
be performed to meet the stringent deadlines encoun- 
tered in broadcast news operations even with late 50 
breaking material. 

Low Resolution Video Browsing and Editing 

[0087] One of the primary features of the system 90 55 
shown in Fig. 10 is the ability to provide a user of the 
journalist workstations 110 with low resolution video to 
allow browsing and editing of the low resolution video to 



ar^^mii 



22 



create storyboaraWJhich may ultimately be used by an 
editor using the video editor 320 to create broadcast 
quality media data. The low resolution editing feature 
allows the journalist to become more involved in the fin- 
ished media product and to incorporate archived media 
data into storyboards without the need for manual 
retrieval of video tapes and operation of a video tape 
player in an edit bay as in previous systems. 
[0088] A journalist, using the journalist workstation 
110, can search the data contained within the library 
server 21 0 of the multimedia archive 200 for low resolu- 
tion video data, audio data and text related to a story 
that the journalist is composing on the journalist work- 
station 110. In response to key search words provided 
by the journalist, the multimedia archive provides a list 
of material contained therein related to the key words. 
The journalist can then select media data for browsing 
or editing on the journalist workstation 1 1 0 from the list 
of material. 

[0089] The graphics user interface for storyboard 
creation provided to the journalist at the journalist work- 
station 1 1 0 is shown in Fig. 1 1 . The user interface 500 
includes a number of windows including a viewing win- 
dow 51 0, a clipnotes window 520, a storyboard window 
530, a storynotes window 540 and a script window 550. 
[0090] The script window 550 provides an area in 
which the journalist can write the main script of a story 
being composed on the journalist workstation 1 1 0. Text 
can be generated in this window using standard word 
processing commands. Graphics, "including painting 
functions, can be performed on the journalist worksta- 
tion 110 and incorporated into the storyboard. 
[0091] The viewing window 51 0 displays a low res- 
olution video component of low resolution media data to 
be viewed and edited on the journalist workstation 110. 
The viewing window also displays the time code 516 of 
the video being displayed, machine controls 518, and 
editing functions such as mark in 512a and mark out 
512b buttons. The machine controls 518 provide con- 
trols for playing a video clip in the viewing window and 
are similar to standard VTR controls. The machine con- 
trols can be selected by the user using a pointing 
device, such as a mouse, or by using special function 
keys on a keyboard of the journalist workstation 110. 
Selecting a clip for display in the viewing window may be 
done by dragging a clip from the storyboard window 530 
(described below) or by selecting a new clip from the 
multimedia archive 200. 

[0092] A second viewing window can be opened on 
the screen at the same time as the viewing window 510. 
The second viewing window, in a preferred embodi- 
ment, is made visible by either shrinking or eliminating 
the storynotes window 540. 

[0093] The mark in button 512a and the mark out 
button 512b are super-imposed in the upper left and 
upper right corners of the viewing window. These but- 
tons are used to perform editing functions at the journal- 
ist workstation 110. When a video clip is being played in 
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the viewing window 510, audio data associated with the 
video data is played on speakers of the journalist work- 
station 1 10. A "video only" or "audio only" indication will 
appear on the video window when the media data being 
displayed or played on the workstation consists of audio 5 
only or video only. 

[0094] The clipnotes window 520 provides a note- 
pad for entry of short notes for each clip viewed on the 
viewing window 510. The storynotes window 540 pro- 
vides an area for the entry of notes that apply to the 10 
whole story to be edited as opposed to the clipnotes 
window 510 which is for notes on individual clips. 
[0095] The storyboard window 530 allows clips and 
subclips to be laid out in sequence. Each of the clips 
532 shown in the storyboard window 530 typically show 75 
the first frame of a corresponding clip, however, the user 
may select a frame, other than the first frame to be 
shown in the storyboard window. The collection of clips 
stored in the storyboard window are referred to as a bin. 
The journalist has the option of playing one of the clips 20 
in the viewing window or playing the bin of clips as 
arranged in the storyboard window. 
[0096] The final pre-edited composition contained 
on the journalist workstation 1 10 may be transferred to 
the multimedia archive 200 for reuse by the journalist or 25 
other journalists on other journalist workstations 110 
and for final editing and playout by a user of the video 
production system 300. 

High Resolution Playout and Editing of Low Resolution 30 
Compositions 

[0097] A composition produced during a low resolu- 
tion activity on a journalist workstation 110 may be 
played out in different ways. A user of a journalist work- 35 
station 110 may play the low resolution composition by 
retrieving the composition data from the multimedia 
archive 200, or a user of the video production system 
300, for example a user of the video editor 320, may 
play and edit a high resolution version of the composi- 40 
tion. The translation of the low resolution composition to 
its high resolution equivalent is transparent to the user 
of the video editor 320. The asset manager 360 using 
registration information of each of the low resolution 
sources used in the composition can identify the equiv- 45 
alent high resolution sources and translate the low res- 
olution composition into its high resolution equivalent. 
Efficient translation by the asset manager 360 requires 
a unique registration system for each of the clips stored 
within the system. Further, the registration method must so 
include means for identifying the corresponding high 
resolution version of low resolution media data. A pre- 
ferred registration method is described in detail further 
below. 

[0098] An editor, using the video editor 320, 55 
receives the high resolution version of the low resolution 
composition created by the journalist, and can further 
edit the composition in broadcast quality format, to pro- 
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vide more precise editin^WTs than accomplished by the 
journalist. 

[0099] In order to provide efficient transmission and 
storage of media data in the system shown in Fig. 1 0, a 
standard file structure is used for the media data con- 
tained within the system. In one embodiment of the 
invention, the media data is organized in a media con- 
tainer 600 as shown in Fig. 1 2. The media container 600 
is divided into five subsections including container data 
610, container timing 620, media security 630, meta 
data 640 and media or media pointers 650. 
[0100] The information contained within the con- 
tainer data 610 describes the container itself and may 
include the following information: the name of the per- 
son that created the container data; the name of the 
person that approved the container data; an identifica- 
tion of the container security; a creation time stamp; the 
name of all people that have modified the data; a modi- 
fication time stamp; a user's Jog; cost information asso- 
ciated with the data; and other user defined elements. 
[0101] Container timing 620 includes information 
related to a relationship over time of the media in the 
container. This information is only applicable to a story 
being prepared for broadcast 

[0102] The media security segment 630 provides 
further information concerning the security level of the 
media contained within the container. This information 
can be used to restrict access to specified personnel of 
media contained within the container. 
[0103] The meta data information describes the 
media stored in the container. In one embodiment, the 
meta data contains the following information for each 
media object in the container: the name of the person 
that approved the data; the name of the person that cre- 
ated the data; a creation time stamp; a media identifier; 
media status; media type; names of all people that have 
modified the data; a modification time stamp; a refer- 
ence number; research descriptors; timing information; 
title; and other user defined elements. 
[0104] The media and media pointers 65 are the 
actual raw data stored in the container. Media objects of 
many types may be stored in a single container. The 
media pointers point to a media object stored in another 
container. By storing a media pointer to another con- 
tainer, rather than the media of the other container itself, 
maximum storage efficiency can be attained throughout 
the system. 

[0105] File structures, other than the container file 
structure described above, may be used for storing the 
media data in the digital multimedia newsroom produc- 
tion system. For example, the Open Media Framework 
(OMF™) file structure, described in Avid Technology. 
Inc. publication OMF™ Interchange Specification, which 
is incorporated herein by reference, may be used as the 
file structure for media files in the system. The file struc- 
ture described in Published PCT Application WO 
93/21636, A Method and Apparatus For Representing 
and Editing Multimedia Compositions, incorporated 
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herein by reference, mi 




be used in embodiments 



of the invention. 
[0106] Another feature of the system shown in Fig. 
1 0 is the ability to uniquely identify the media objects 
stored within the system and to locate other versions of 5 
media data that correspond to the media objects. The 
ability of the system 90 to locate a high resolution ver- 
sion of media data, corresponding to a low resolution 
version of the same media data, allows the asset man- 
ager 360 to provide a high resolution translation of com- w 
binations or storyboards generated by the journalist 
workstation 110, such that the translation is transparent 
to an editor using the video editor 320. 
[0107] The asset manager can uniquely identify the 
low resolution and high resolution media data in a 15 
number of ways. In one embodiment of the invention, 
the media data, when captured by the media recorder 
330, is assigned a unique time code stamp correspond- 
ing to the date and time that the media data is captured 
by the media recorder 330. Using this scheme, the low 20 
resolution version of the media data and the high reso- 
lution version of the media data is assigned the same 
identification number. However, since the low resolution 
media data is stored in the multimedia archive 200, and 
the high resolution media data is stored in the media 25 
server, there is no opportunity for confusion between 
the versions of the media data. The asset manager, in 
translating a combination or storyboard from a low res- 
olution version to a high resolution version, can locate 
the high resolution version of each media object of the 30 
combination in the media server based on the identifica- 
tion number of the corresponding low resolution version 
of the media object. The above-described media data 
identifying method is not preferred for use at broadcast 
locations that do not maintain a unique timecode stamp. 35 
[0108] In one embodiment of the invention, the 
asset manager 360 may be implemented using Media 
File Manager (MFM) and Source Manager (SM) soft- 
ware as described in U.S. Patent No. 5,267,351 to 
Reber et al which is incorporated herein by reference. 40 
This software provides a unique identifier to media data 
captured by the system and maintains a table of rela- 
tionships between media objects contained within the 
system such that the asset manager 360 can identify a 
corresponding version of low resolution or high resolu- 45 
tion media data. 

[0109] In an alternate embodiment of the invention, 
a digital multimedia newsroom production system con- 
sists only of the core newsroom system 100 and the 
multimedia archive system 200 coupled by the digital so 
network 400. In this alternate embodiment a low reso- 
lution capture device is coupled to the network 400 to 
capture low resolution media data for storage in the 
news servers 120 and the multimedia archive system 
200. In this embodiment, the journalist workstations 110 55 
provide the full storyboard functions described above 
with respect to the system 90 shown in Fig. 1 0. 
[0110] Embodiments of the invention overcome lim- 



83 A2 26 

itations of prior^lKystems by providing a fully inte- 
grated digital multimedia newsroom. In embodiments of 
the invention, a journalist in a newsroom may create a 
multimedia storyboard of a news story which is elec- 
tronically transferred over a digital network to an editing 
and production system for final editing and broadcast to 
air. 

[0111] Embodiments of the invention have been 
described with respect to a multimedia production sys- 
tem in a newsroom environment, however, embodi- 
ments of the invention are not limited to a newsroom 
environment, but rather may be used in other multime- 
dia environments as well, such as radio, and in the pro- 
duction of entertainment programming. 
[0112] In embodiments of the invention described 
above, the multimedia data processed on the journalist 
workstation 110 has been described as low resolution 
multimedia data. The user interface provided by the 
journalist workstation 110 may also be used to create 
storyboards using high resolution multimedia data. 
[0113] Furthermore, the embodiments have been 
described in a newsroom context. However, the inven- 
tion may be applied anywhere in the movie, television 
and cable industry, where multimedia data, and particu- 
larly, motion video data, is to be processed. In particu- 
lar, the invention is suitable for active movie systems, 
video conferencing, and cable pay per view systems. 
[0114] Having thus described at least one illustra- 
tive embodiment of the invention, various alterations, 
modifications and improvements will readily occur to 
those skilled in the art. Such alterations, modifications 
and improvements are intended to be within the scope 
and spirit of the invention. Accordingly, the foregoing 
description by way of example only, it is not intended as 
limiting. The invention's limit is defined only in the claims 
and the equivalents thereto. 

[0115] This application is divided from European 
Patent Application No. 97920345.2, the content of 
which was published as WO-A-97/3941 1 , the entire 
contents of which are incorporated herein by reference. 
Further aspects and preferred features, corresponding 
to the original claims of the parent application are set 
out below in the following numbered clauses. 

1 . A multimedia system, comprising: 

a multimedia capture and encoding system that 
captures multimedia data, and substantially 
simultaneously provides a first compressed 
version of the multimedia data having a first 
resolution, and a second compressed version 
of the multimedia data having a second resolu- 
tion that is different than the first resolution; 
a multimedia storage system, coupled to the 
' multimedia capture and encoding system, that 

stores multimedia information including the first 
and second compressed versions of the multi- 
media data; and 
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a video editing and playback system coupled to 
the multimedia storage system, the video edit- 
ing and playback system including: 

editing circuitry that generates a composi- s 
tion that uses a portion of the first com- 
pressed version, and 
. playback circuitry that plays the composi- 
tion using a portion of the second com- 
pressed version that corresponds to the to 
portion of the first compressed version. 

2. The multimedia system of clause 1, wherein the 
editing circuitry is disposed on a first video station; 
wherein the playback circuitry is disposed on a sec- is 
ond video station; and wherein the video editing 
and playback system further includes a network 
connection disposed between the first and second 
video stations to transfer the composition from the 
first video station to the second video station. 20 

3. The multimedia system of clause 2, wherein 
each of the first and second video stations includes 
a screen; and wherein the first and second resolu- 
tions are time resolutions so that the screen of the 25 
first video station displays frames of the first com- 
pressed video with a first time latency and the 
screen of the second video station displays frames 

of the second compressed video with a second time 
latency that is different than the first time latency. 30 

4. The multimedia system of clause 2, wherein 
each of the first and second video stations includes 
a screen; and wherein the first and second resolu- 
tions are spatial resolutions so that the screen of 35 
the first video station displays frames of the first 
compressed video with a first image clarity and the 
screen of the second video Station displays frames 

of the second compressed video with a second 
image clarity that is different than the first image 40 
clarity. 

5. A multimedia system, comprising: 

a multimedia capture and encoding system that 45 
captures multimedia data, and substantially 
simultaneously provides a compressed version 
of the multimedia data having a first resolution; 
and 

a multimedia storage system, coupled to the so 
multimedia capture and encoding system, that 
stores multimedia information including the 
compressed version of the multimedia data, 
and provides to a network the compressed ver- 
sion of the multimedia data substantially simul- 55 
taneously as the compressed version is stored. 

6. The multimedia system of clause 5, wherein the. 
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multimedia storage ^PTem includes a server cou- 
pled to the network, that sends the compressed 
version of the multimedia data on the network in 
multicast form. 

7. The multimedia system of clause 5, wherein the 
multimedia storage system includes a server cou- 
pled to the network, that sends the compressed 
version of the multimedia data on the network in 
response to a request from a video host. 

8. The multimedia system of clause. 7, further com- 
prising: 

a video host coupled to the network, that sends 
a first request to the server for a first portion of 
the compressed version of the multimedia 
data, determines an amount of time to wait 
based on a length of the first portion and a 
response time of the first request, and sends a 
second request to the server for a second por- 
tion of the compressed version of the multime- 
dia data after waiting the determined amount of 
time. 

Claims 

1. A multimedia system comprising: 

means for capturing multimedia data and pro- 
viding an encoded version of the multimedia 
data; 

means for providing to a network at least a por- 
tion of the encoded version of the multimedia 
data; and 

means for substantially simultaneously storing, 
in a file, the encoded version of the multimedia 
data. 

2. The multimedia system of claim 1, wherein the 
means for providing to a network at least a portion 
of the encoded version of the multimedia data com- 
prises means for providing to a network at least a 
portion of the encoded version of the multimedia 
data without having to read the at least a portion of 
the multimedia data from the file. 

3. The multimedia system of claim 2, wherein the 
means for providing to a network at least a portion 
of the encoded version of the multimedia data com- 
prises means for reading the at least a portion of 
the encoded version of the multimedia data from 
the file and means for providing the at least a por- 
tion of the multimedia data read from the file to the 
network. 

4. The multimedia system of claim 1, wherein the 
means for providing to a network includes means 
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for sending the at l^^^ portion of the encoded 
version of the multimedia data on the network in 
multicast form. 

5. The multimedia system of claim 1, wherein the 
means for providing to a network includes: 

means, coupled to a network, for receiving a 
request from a video host; and 
means, coupled to the network, for sending the 
at least a portion of the encoded version of the 
multimedia data to the video host in response 
to the request. 

6. The multimedia system of claim 5, further compris- 
ing a video host, the video host comprising: 

means for sending a first request for a first por- 
tion of the encoded version of the multimedia 
data; 20 
means for determining an amount of time to 
wait based on a length of the first portion and a 
response time of the first request; and 
means for sending a second request for a sec- 
ond portion of the encoded version of the mul- 25 
timedia data after waiting the determined 
amount of time. 
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network at leWPB portion of the encoded version of 
the multimedia data comprises reading the at least 
a portion of the encoded version of the multimedia 
data from the file and providing the at least a por- 
5 tion of the multimedia data read from the file to a 

network. 

12. The method of claim 9, wherein providing to a net* 
work includes sending the at least a portion of the 

w encoded version of the multimedia data on the net- 
work in multicast form. 

13. The method of claim 9, wherein the providing to a 
network includes: 
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7. The multimedia system of claim 1, wherein the 
means for storing includes means for appending to 
the encoded version of the multimedia data written 
in the file. 

8. The multimedia system of claim 1, wherein the 
means for storing the encoded version of the multi- 
media data to the file comprises means for storing 
the encoded version of the multimedia data to a 
computer data file on a computer readable and wri- 
table medium. 

9. A method for use in a multimedia system compris- 
ing: 

capturing multimedia data and providing an 
encoded version of the multimedia data; 
providing to a network at least a portion of the 
encoded version of the multimedia data; and 
substantially simultaneously storing, in a file, 
the encoded version of the multimedia data. 

10. The method of claim 9, wherein the providing to a 
network at least a portion of the encoded version of 
the multimedia data comprises providing to a net- 
work at least a portion of the encoded version of the 
multimedia data without having to read the at least 
a portion of the multimedia data from the file. 

11. The method of claim 10, wherein the providing to a 
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receiving a request for the at least a portion of 
the encoded version of the multimedia data; 
and 

sending the at least a portion of the encoded 
version of the multimedia data in response to 
the request. 

14. The method of claim 13, further comprising: 

sending a first request for a first portion of the 
encoded version of the multimedia data; 
determining an amount of time to wait based 
on a length of the first portion and a response 
time of the first request; and 
sending a second request for a second portion 
of the encoded version of the multimedia data 
after waiting the determined amount of time. 

15. The method of claim 9, wherein the storing includes 
appending to the encoded version of the multime- 

. dia data written in the file. 



16. The method of claim 9, wherein the storing the 
encoded version of the multimedia data to the file 

40 comprises storing the encoded version of the multi- 
media data to a computer data file on a computer 
readable and writable medium. 

17. A system for reading multimedia data over a net- 
45 work, the system comprising: 

means for requesting from over the network a 

portion of the multimedia data; 

means for determining a response time for 

so receiving the portion; and 

means for requesting a further portion of the 
multimedia data from over the network accord- 
ing to the. response time for receiving the por- 
tion and a desired playback rate of the received 

55 multimedia data. 

18. The system of claim 17, wherein the means for 
requesting from over the network a portion of the 
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multimedia data includes means for requesting 
from a file system over the network a portion of at 
least one file in which an encoded version of multi- 
media data is stored, and wherein the means for 
requesting a further portion of the multimedia data 
from over the network includes means for request- 
ing a further portion of the at least one file from the 
file system over the network according to the 
response time for receiving the portion and a 
desired playback rate of the received multimedia 
data. 

19. The system of claim 17, wherein the multimedia 
data comprises video data. 

20. The system of claim 17 wherein the means for 
requesting a further portion includes: 

means for determining an amount of time to 
wait according to the response time for receiv- 
ing the portion and a desired playback rate of 
the received multimedia data; and 
means for requesting the further portion after 
waiting the determined amount of time. 

21. The system of claim 17, wherein the means for 
requesting a further portion includes: 

means for determining a time for requesting the 
further portion based at least in part on the 
response time for receiving the portion and an 
amount of time needed to be through playing 
the portion. 

22. The system of claim 21 wherein the means for 
requesting a further portion further includes means 
for requesting the further portion at the determined 
time for requesting the further portion. 

23. The system of claim 17 further including: 

means for prompting, while receiving and stor- 
ing, a user to select any one or more portions 
of the encoded version of the multimedia data 
stream from the file; 

means for receiving an indication of the user's 
selection of one or more portions of the 
encoded version of the multimedia data from 
the file; and 

means for interactively playing, while receiving 
and storing, the selected one or more portions 
of the encoded version of the multimedia data 
from the file for the user. 

24. The system of claim 23, wherein the means for 
receiving a multimedia data stream and storing 
includes means for receiving a plurality of multime- 
dia data streams and storing a corresponding plu- 



rality of encoded ^Wsions of multimedia data 
streams in a corresponding plurality of files, and 
wherein the means for prompting includes means 
for prompting, while receiving and storing, a user to 
5 select any one or more portions of the encoded ver- 

sions of the multimedia data streams from the files, 
and wherein the means for receiving an indication 
includes means for receiving an indication of a 
user's selection of any one or more portions of two 
10 or more of the encoded versions of the multimedia 

data streams from the files, and wherein the means 
for interactively playing further includes means for 
interactively playing, while receiving and storing, 
the selected one or more portions of the two or 
more of the encoded versions of the multimedia 
data streams from the files. 

25. A method for reading multimedia data over a net- 
work, the method comprising: 
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requesting from over the network a portion of 
the multimedia data; 

determining a response time for receiving the 
portion; and 

requesting a further portion of the multimedia 
data from over the network according to the 
response time for receiving the portion and a 
desired playback rate of the received multime- 
dia data. 

26. The method of claim 25, wherein the requesting 
from over the network a portion of the multimedia 
data includes requesting from a file system over the 
network a portion of at least one file in which an 
encoded version of multimedia data is stored, and 
wherein the requesting a further portion of the mul- 
timedia data from over the network includes 
requesting a further portion of the at least one file 
from the file system over the network according to 
the response time for receiving the portion and a 
desired playback rate of the received multimedia 
data. 

27. The method of claim 25, wherein the multimedia 
data comprises video data. 

28. The method of claim 25 wherein the requesting a 
further portion includes: 

determining an amount of time to wait accord- 
ing to the response time for receiving the por- 
tion and a desired playback rate of the received 
multimedia data; and 

requesting the further portion after waiting the 
determined amount of time. 

29. The method of claim 25, wherein the requesting a 
further portion includes: 
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determining a ^^por requesting the further 
portion based at least in part on the response 
time for receiving the portion and an amount of 
time needed to be through playing the portion. 

5 

30. The method of claim 29 wherein the requesting a 
further portion further includes requesting the fur- 
ther portion at the determined time. 

31. The method of claim 25 further including: w 



prompting, while receiving and storing, a user 
to select any one or more portions of the 
encoded version of the multimedia data stream 
from the file; 75 
receiving an indication of the user's selection of 
one or more portions of the encoded version of 
the multimedia data from the file; and 
interactively playing, while receiving and stor- 
ing, the selected one or more portions of the 20 
encoded version of the multimedia data from 
the file for the user. 



32. The method of claim 31, wherein the receiving a 
multimedia data stream and storing includes receiv- 25 
ing a plurality of multimedia data streams and stor- 
ing a corresponding plurality of encoded versions of 
multimedia data streams in a corresponding plural- 
ity of files, and wherein the prompting includes 
prompting, while receiving and storing, a user to 30 
select any one or more portions of the encoded ver- 
sions of the multimedia data streams from the files, 
and wherein the receiving an indication includes 
receiving an indication of a user's selection of any 
one or more portions of two or more of the encoded 35 
versions of the multimedia data streams from the 
files, and wherein the interactively playing further 
includes interactively playing, while receiving and 
storing, the selected one or more portions of the 
two or more of the encoded versions of the multime- 40 
dia data streams from the files. 
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